PiOsu
Osu on Raspberry Pi
Aaron Peng (ahp67), Andrew Cai (ayc57)
Objective: Implement all basic Osu features on Raspberry Pi so can be played with touch screen. Incorporate automatic beatmap loading. Incorporate scoring and difficulty adjustment.
Demonstration Video
Introduction
Throughout the semester we learned how to use Pygame, a powerful library for creating games and other graphical applications. We implemented all the base features, including single note rhythms and legato rhythms (connected note sliders) for running on the Raspberry Pi PiTFT. Our game also features scoring and automatic beatmap generation.
Design
Our design does not involve any additional hardware elements besides the PiTFT, so we will discuss the design of the software elements.
Menu
The start screen of the menu features a large start button that follows the aesthetics of the gameplay.
The next page features a scrolling menu for level selection.
Most of the aspects of the menu design are basic, however implementing scrolling options had some increased complexity compared to a static menu. The code below documents how scrolling is handled. While the mouse button is down (or finger is pressed down), the menu items move up and down in the opposite direction to the finger movement. This makes for natural movement when using a touch screen. There are also checks to prevent scrolling the menu off screen.
The font size is dynamically scaled, so the menu item inline with the white triangle pointer is the largest. The below code scales sizes based on distance from the pointer.
To make the menu more intuitive, the items are always snapped to align with the pointer after the user adjusts their positions. The code snippet below handles this.
Game Mechanics
Example Measure
The bulk of the code is in implementing the game mechanics. All the mechanics of the game are structured around measures. A measure is a collection of numbered rhythm elements (notes). A measure start specifies the time offset in relation to the start of the song where the measure starts. All the rhythm elements within the measure are timed relative to the measure start point with an offset. Each rhythm element also has a duration argument that specifies how early the note will appear. This allows for parameterizing the difficulty, since the earlier a note shows up the easier it is for a player to react to it. The absolute time since the start of the song that the user should tap the note is given by measure offset + note offset + duration. The note also has an attribute that controls the length fade animation that activates when a user successfully executes the mechanic.
To help provide a visual indicator of when the beat is coming, there is also a shrinking approach circle. which we call a halo. The approach circle gradually shrinks at a constant rate since the appearance of the note until the time the note should be tapped. Code snippets below explain the basic implementation of the Note element. The code is also included inside the code appendix at the bottom of the page if the reader wishes to understand more.
The note element is implemented with Pygame Sprites. Sprites provide a convenient interface that allows us to update the appearance and state of the rhythm elements and then draw (render) all the elements at once. It is necessary for us to modify the appearance of a note throughout it’s life cycle to support the shrinking halo visual. Dirty Sprites, a subclass of the normal Sprite, are used for efficiency. The appearance of a note doesn’t always change each clock tick - Dirty Sprites let us indicate whether or not the sprite needs to be redrawn by setting the value of the dirty attribute. The approach circle is implemented as the Halo class and is heavily dependent on the note itself which is implemented as the Note class. At a high level, a Note instance functions like a finite state machine that proceed through up to 7 different states.
Basic Note states
The snippets below illustrate how we update the sprite. If the time to display the note has not yet arrived, it remains in the UNSTARTED state. Once the note is STARTED, the halo radius decreases until it is the same size as the note. An instance of the HALO class accompanies each note, reading the updated radius value, and re-rendering the approach circle each game tick. Inside the STARTED state, there is also a tick buffer that gives some leeway to the player to click the note early (since no one can perfectly click the note on the exact game tick). If the tap time comes before this buffer, the note moves into the FAILED state instead. Otherwise, it moves into a WAITING state when the halo and note radius are the same. Inside the WAITING state, we give a late tick buffer that specifies the window of time in which it is valid to tap the note after the halo and the note are the same size. If the window is surpassed, the note moves into the FAILED state automatically. On a successful click, the note moves into the SUCCEEDED state, triggering a fade animation. Otherwise, if the note enters the FAILED state, the note and halo are replaced with a red “X” to signal the player’s failure (class function show_fail()). After either of these animations conclude, the note retires to the FINISHED state and awaits cleanup.
Legato rhythm elements follow much the same logic as the Note elements, with a few twists. Notably, they accept a collection of screen coordinates as an input to define the track that players must slide the circle along to successfully complete the mechanic. While we could manually pick out the points in the path, we thought it would be a lot cooler to automatically generate smooth paths using splines. Here is the Spline class we implemented:
The CubicSpline() method from Scipy.Interpolate is the lynchpin here, solving a series of boundary constraints in order to generate a continuous polynomial function defining a path that joins the input waypoints. However, the function only determines one control and we have two degrees of freedom (x coordinate and y coordinate). Hence, our custom Spline class initializes two splines, spline_x and spline_y. The other point of interest is the discrete() class method, which converts the continous spline functions into a set of discrete points. This step is necessary since we can only draw pixels at integer screen coordinates.
After splines are generated, many aspects of note elements carry over for the legato elements. For the first phase of the legato rhythm, the same timing mechanics apply- just like notes, the player must click the legato note when the approach circle converges. However, the player must then drag the note to the end of the displayed path. We added an additional state called SLIDING to encapsulate the new logic. While there are no hard time limits on when the player must drag the note to the end of the legato path, taking too long will cause the player to miss the next note. The player will also fail the legato rhythm if they drag the note too far away from the cener of the path. Hence, the legato mechanic challenges both the speed and precision of the players taps.
On a side note, to programmatically render the spline paths on the screen, we first get an array reference to the screen using pygame.surfarray.pixels2(d). With the reference in hand, we set the pixel color of the array points within a certain radius of the discretized spline to the desired path color.
Since making beatmaps by hand is tedious, we utilize the many publicly available beatmaps that are created by members of the Osu community. Osu beatmaps feature a human readable file format .osu that contains information about when and where each timing element should be clicked. Using this information and the measure/note classes described previously, we can automatically generate beatmaps.
Our current level generator is randomized, and generates measures containing a random number of notes between 1 and 4. It also has random locations in 9 different quadrants on the screen where a note may be placed. The note duration can be randomized, or defined. Currently, it is simply defined as 1 second for all notes. The generator similarly creates a randomly parameterized legato element with a 20% probability. It is important to clarify that the algorithm does not randomize when the rhythm elements need to be clicked - they will always fall on a point in time defined in the beatmap. The function returns a lazily-executed iterator, as shown below, because the large measure lists that define full beatmaps crash Python.
The following snippet shows the gamer driver loop that loads measures in groups of two, pausing the music briefly while loading to prevent music from desyncing from the beatmap. This small pause is not heard, but over the course of many measures prevents desync. In addition to loading measures into memory, the main duties of the loop include updating the elapsed time, signalling measure objects to start displaying notes when the measure offset time is surpassed, cleaning up completed measures, accumulating the score, and returning to the main menu when the song concludes.
Testing
To test, we played through the beatmaps on the Pi to check if the generated times seemed to be on the beat. We also compared our results to the actual beatmap on the real desktop Osu. We also have additional testing with a test beatmap to check basic mechanics.
Result
In the end, we got all the basic Osu game mechanics working on the PiTFT. If we had more time, there are many additional improvements that can be made, such as utilizing PiCameras to incorporate some sort of gesture detection system for scoring at certain points in the song. There is also the ability to parameterize level difficulty by adding flags to specify how close the automatic beatmap generation places the next note to the previous one, and adjust approach rate to give users more or less time to react to a note.